1. Field of the Invention
The present invention generally relates to a shadow detection method and device, and more particularly relates to a method and device for detecting and removing shadow by utilizing a color image/grayscale image and depth map/disparity map captured by a two-lens camera or stereo camera.
2. Description of the Related Art
Shadow is a kind of familiar natural phenomenon, and often exists in a color image or grayscale image captured by a well-used camera. FIGS. 1A and 1B illustrate that shadow exists in a grayscale image and depth image captured by a two-lens camera, respectively. The existence of shadow brings a lot of difficulties and restrictions to computer-based image processing. In particular, in object detection performed on an image, the existence of shadow may negatively influence the accuracy of the object detection. For example, a segmentation method is easily influenced by shadow, thereby generating a segmentation error. In addition, an object detection algorithm or object tracking algorithm is also easily influenced by shadow, thereby generating an incorrect detection result or incorrect tracking result. As a result, a shadow detection and removal technique used for a video or single-frame image has attracted attention.
Well-used shadow removal methods are carried out on the basis of a color image or grayscale image, for example, an intelligent edge matching method, a texture-based method, or a color-and-texture-based method. However, these kinds of methods may be dramatically influenced by environmental background complexity, and may be easily influenced by lighting conditions.
In U.S. Pat. No. 8,294,794 B2, a shadow removal method used for a vehicle-mounted camera is disclosed. In this method, an edge of an input image is recognized, and at same time, a corresponding edge of a corresponding illumination-invariant image is recognized. By respectively determining the existence states of the two edges in the two images, it is determined whether shadow exists. This method is mainly based on an edge detection technique. In this method, an input image and its corresponding illumination-invariant image are utilized at the same time for detecting edges, respectively. On the basis of the difference between the detected edges, it is determined whether shadow exits. However, in this method, it is necessary to obtain the corresponding illumination-invariant image, and the removal of shadow is restricted by the obtention of the corresponding illumination-invariant image and the quality of the corresponding illumination-invariant image.
In U.S. Pat. No. 7,133,083 B2, a dynamic shadow removal method is disclosed in which at least one camera, one projector, and one screen are utilized. A spatial relationship among the camera, the projector, and the screen is calculated, and then by calculating the difference between a projected image and an image captured by the camera, it is determined whether shadow exists. In this method, it is necessary to use at least one camera, one projector, and one screen, and it is also necessary to build the spatial relationship among the camera, the projector, and the screen. The main idea of this method is building a predicted image according to projected contents, then comparing the predicted image and an actually-captured image, and then, on the basis of the difference of the two images, determining whether shadow exists. However, in this method, it is necessary to know the projected contents in advance. Furthermore, the application of this method is very limited; for example, this method is only suitable to be used in shadow detection of a projection exhibition system.
Moreover, in a doctoral thesis entitled “Detection, Tracking, and Identification of People using Stereo Vision” written by Yong Zhao of Brown University, a shadow removal method using background models on the basis of a depth map is disclosed. In this method, two background models are used, namely, a depth background model on the basis of a depth map as well as an appearance background model on the basis of a RGB/grayscale image. First, on the basis of the appearance background model, appearance foreground is detected which probably includes shadow. Second, it is determined whether the appearance foreground overlaps with the depth background model. If the appearance foreground overlaps with the depth background model, then that means the overlapped part is shadow. The main idea of this method is that shadow cannot change the depth of background. However, this method relies on the quality of the two background models too much, and it is difficult to acquire a dense depth background model by utilizing a well-used two-lens camera. As a result, this method is not valid in some cases.